Dear Dr. Bauder and Community Members,I have rewritten my project for the final time and I am hoping it includes everything on the rubric which I will also post to this Wiki for eveyrone to refer to. I have added all of the required categories and I have already created the Quick Guide for Testing Coordinators. I have decided to do a weries of podcasts with the publicaiton. The Idaho Training Clearinghouse can't support video uploads of more than a few mintues and the Quick Guide is more than twenty pages of Snag It screen shots of every single function. I am going to try to post it here but am not sure if the site will accept a file this size. Thank you all for your very helpful feedback and assistance! I have enjoyed reading all of your projects and getting to know you!The project is geting so big I just uploaded this link to the full paper and the Quick Guide which I wrote for your convenience rather than cut and paste onto the page. Just click on the Module or Quick Guide to open in Word.. I will be recording the Podcast this week so any feedback would be appreciated so I don''t have to redo it:)Thanks for looking!
With WArm Regards,CarolPS My orginal project is below this new verision What is below is the older versions one on top of the other. I have rewritten the project based upon feedback from you and from Dr. Bauder on the module 3 assignments. I appreciate all of your suggestions and have incorporated them into this revision! So there are two verisons the revised one is first on this page and the module 3 version follows. Thank you for your input I look forward to reading your responses! With Warm Regards, Carol Modules One Through Four Summary
Organizational History and Background
Name of Organization and Project
Organization - Idaho State Department of Education – Division of Assessment Project - Training Key Personnel for I-PASS (Individual Portfolio Artifact Submission System)
Project Summary This project involves the training of key personnel for I-PASS (Individual Portfolio Artifact Submission System). I PASS is an online system which was created as a collaborative venture with Caring Technologies of Boise and the Idaho SDE department over the course of the last year to deliver the first in the nation, alternate assessment via a digital online portfolio. I-PASS allows teachers to upload evidence of student learning via video, digital photo or scanned or faxed student work into individual portfolios for each content area including Math, Reading, Language Usage and Science. This online assessment was developed to replace a former teacher rating scale which consisted of a single page checklist for each content area. This digital portfolio was launched in May and there are more than 70,000 individual pieces of student work uploaded into the I PASS system. Four hundred Flip cameras were purchased for teachers across the state to document student learning via video. Educators have already received training via Webinar and Podcast on the use of the system as well as how to administer the assessment itself. All training and materials are available on the Idaho Training Clearing House. An Alternate Assessment Learning Community is maintained there as well where FAQ, The scoring of these portfolios has just been completed online by teachers throughout the state, whom I have trained in a series of four regional workshops.
Problems There are three identified problems. The first is, that there will be no Idaho State Department of Education Information Technology support helpdesk staff to provide online or telephone support to districts this Fall on the I-PASS system. It is expected that responsibility for maintaining classes and providing support to teachers and staff will be transferred to the district level. The second identified problem is related to the first. The district level IT and Testing Coordinators have not been trained to perform the support tasks necessary to assist district teachers statewide on the use of the I-PASS system. The third problem is that the training and delivery has not be created for this transfer of responsibility, as this is a newly developed system we created ourselves so does not come with documentation or support materials for training. It has become evident that a support structure needs to be developed on a state-wide and local level for teachers through these key personnel at the district level. District IT staff and Testing Coordinators will be required to maintain the system at the local level. Training will need to be done so that they have the skills necessary to maintain online class lists, updating them as needed with the correct content, setting up the hardware, networked software and computer access for teachers to use to conduct the assessment.
3. Background of Organization When was the organization founded? Where is it located? The Idaho State Department of Education Division of Assessment was just organized this past summer as a new division within the Idaho State Department of Education. It was formerly housed within the Idaho State Board of Education. It is housed at the SDE in Boise, Idaho. Two new coordinators and a new Deputy Superintendant were hired coming on board in July and August. It is a brand new division working together for one year. What is the mission or vision of the organization? The Division of Assessment has not developed or published an official mission statement. Here is what is on the web. The Division of Assessment will measure and improve student achievement, identify areas needing intervention, remediation or acceleration, inform parents and guardians about their child's progress, provide comparative local, state and national data regarding the achievement of students in essential skill areas, identify performance trends in student achievement across grade levels tested and student growth over time, and help to determine the technical assistance and consultation priorities for the State Department of Education. What are the stated goals of the organization? The Division of Assessment has not developed or published goals. Unofficially the goal is to provide for the administration of all assessments including federally mandated as well as state mandated assessments to all students including ELL and special education students according to the NCLB and IDEA federal guidelines and laws. Who is the organization’s clientele? The identified clientele of the Department of Education includes; students, educators and administrators throughout the state of Idaho. The specific identified clientele for the I PASS project are those students with the most significant cognitive disabilities and their teachers. This student population is 1% of the total population of special education students at approximately 1,500 students who have been identified on their IEP as eligible to take the Idaho Standards Achievement Tests Alternate (ISAT – Alt). This is the federally mandated NCLB and IDEA state assessment corresponding to the ISAT itself designed for the general education population. Who are the key personnel in the organization that can provide you with insights into the performance problem for which you will propose an intervention? The key personnel with insight into the performance issue, is the Deputy Superintendent of the Assessment Division. Final decisions regarding state wide training and service are made in collaboration with her, After discussion with the Deputy Superintendent, it was reported that IT would no longer be able to provide support directly to district teachers. The SDE would provide the initial data entry of district provided demographics such as student, teacher, assessment specifics at the beginning of the school year but that districts would have to maintain class lists including adding, removing students and teachers, associating specific assessment folders with students and providing technical assistance for uploading data. The Testing Coordinators and IT staff, at the district level, also provided the identification of the problem as well as for the justification for the choice of solution. Rather than poll these staff regarding their individual needs, email records were reviewed indicating that over 8,000 emails were received from educators requesting help with I PASS by state IT department personnel. In addition the district level IT and Testing Coordinators have expressed a need for further training and support directly to the Deputy Superintendent. The results of the interview also revealed that there is no funding to hire another administrative level person at the SDE assessment or IT division levels.. Does the organization have a Web site? If so, record the URL. http://www.sde.idaho.gov/ Do you have access to any public relations materials or annual reports for the organization? Where can you obtain these? http://www.sde.idaho.gov/ General website for State Department of Education http://www.sde.idaho.gov/site/assessment/ISAT/results.htm ISAT test results for Idaho http://itcnew.idahotc.com/dnn/alternate-assessment.aspx Idaho Training Clearinghouse where I maintain a Learning Community and publish all WebEx, podcast and publications
I-PASS (Individual Portfolio Academic Submission System) Support Project Human Performance Technology (HPT) attempts to initiate changes to a system to improve it in terms of the achievements that have been identified as critical to success. In utilizing the HPT model on the implementation of I PASS and in the review of the success of the previous year, the focus will be on these outcomes and accomplishments which have been identified as critical to performance. The procedures identified to ameliorate the problems will be applied in a methodical manner in identifying the performance gap, indentifying appropriate interventions and in their application to improve performance. (Stolovitch & Keeps 1999).
The identified problems are; There are three identified problems. The first is, that there will be no Idaho State Department of Education Information Technology support helpdesk staff to provide online or telephone support to districts this Fall on the I-PASS system. It is expected that responsibility for maintaining classes and providing support to teachers and staff will be transferred to the district level. The second identified problem is related to the first. The district level IT and Testing Coordinators have not been trained to perform the support tasks necessary to assist district teachers statewide on the use of the I-PASS system. The third problem is that the training and delivery has not be created for this transfer of responsibility, as this is a newly developed system we created ourselves so does not come with documentation or support materials for training. District Testing coordinators do not know how to create class lists, assign tests and students so the SDE gets a lot of help desk referrals to correct errors or make changes as students move and change schools, teachers and districts. A statewide information system is in place to establish the initial class, school and district list but is riddled with data errors which must be corrected manually as they arise. The data is only as good as the accuracy of the person who enters it and there are bound to be errors. The volume of these data errors and constant changes in assignment of individual students is so extensive despite expert assistance, the capacity of the SDE helpdesk hat been stretched to its limit and will not be able to continue to handle this increased demand. There will be a push out to districts for IT and Testing Coordinators to maintain these needed adjustments to the initial data uploaded in October.
I-PASS (Individual Portfolio Academic Submission System) Support Proposed Solutions The low-cost strategy (at a cost of approximately 0 - $5,000) is to produce a webinar and archive it on the Idaho Training Clearinghouse, which trains every testing coordinator in the state of Idaho (120) how to maintain the records for the ISAT-Alt (Idaho State Assessment Tests Alternate) using the I-PASS online digital portfolio system. The components of this training would be conducted via Webinar training then demo of the online system live followed by a Survey Monkey assessment. The cost would be included in the salary already paid for my position so in essence would be zero additional cost. Another low cost solution would be to create a NING to deliver a webinar, training materials as well as video. The advantage to developing a NING is it would provide for a place for IT folks to meet in a learning community to share solutions and information amongst themselves. Tech Talk – Another free format for communicating with IT district personell is through the portal Tech Talk. It provides for a platform for blogging about various subjects and is a secure site providing for a free forum for concerns and information without providing public access. The most practical cost effective solution for these problems may be to use a combination of the two free services; Idaho Training Clearinghouse and Tech Talk. Neither provides for all of the necessary components for great training but together may be adequate to get the job done in tough economic times. NING just announced that it will charge beginning this month so this option is probably not going to work out but probably would provide the best platform. Necessary Training for Testing Coordinators Includes;
Assigning Content Area Assessment Folders to Students
Assigning Students to Teachers, Schools and Districts
Assigning Teachers to Schools and Districts
Monitoring these Assignments for Change
Maintaining excel records of all students eligible to take the Alternate Assessment
Working with teachers to upload student data. Maintaining and providing the necessary hardware and software for I PASS
Troubleshooting simple technology issues to assist teachers.
Monitoring the upload of data to assure assessment completion of eligible students. The Identified Possible Solutions In the application of the HPT model “procedures derived from scientific research and professional experience to the solution of practical problems”. (Stolovitch & Keeps 1999). HPT is systematic in that it is an organized approach to apply scientific research methodology and results to identify, define and analyze performance gaps then apply the interventions in a methodical manner to achieve the desired improvement in human performance of both the Assessment Coordinator and the Testing Coordinators as well as the IT personnel at the district level. apply them in a controlled manner. Documented evidence will be created via Survey Monkey both prior to implementation and subsequent to it to identify areas of need, knowledge base and potential solutions then to analyze results of the applied t raining solutions. The medium-cost strategy would resolve the performance gap over several years’ time (at a cost of approximately $25,000). This medium cost strategy would be a combination of online delivery of training and support and live face to face. This would be accomplished by bringng the testing coordinators and IT professionals to Boise for hands on training once per year on the same topics rather than have them view webinars on the Idaho Training Clearinghouse, or participate in a NING and Tech Talk remotely . This would provide a hands-on experience where they could ask questions and receive assistance in addition to live demos. It would give them opportunities to network too as they are often isolated in remote districts and do not have access to a live learning community. The high-cost strategy “magic wand” to eliminate the performance gap cost of approximately $50,000). This high cost strategy could be to hire a state department staff person dedicated to IT and Testing Coordinator Support specifically assigned to maintaining the I PASS system, Another high cost strategy would be to provide funds for the SDE to implement a state-wide help desk tracking system accessible to teachers, testing coordinators and It personnel across the state. This would provide support and training at the state and district level. They would handle training and maintaining the system and their position would separate out the duties of from training on assessment and supporting teachers in the classroom and the maintenance of the actual system IT and Testing Coordinator support itself. I don't think it is feasible. That's why it is called a magic wand solution! It is make believe. It is the expectation in tough economic times to do more with less not less with more. Throughout this process of designing training to ameliorate the indentified problems the Human Performance and Technology model will be utilized to deal with the performance of the Alternate Assessment Coordinator, district information technology personnel and the district level testing coordinators. The effort will be to search for the most effective and efficient way to improve performance but accomplish this in the most cost-efficient way The focus will be to”enhance individual human performance by looking at the organization, the process and the individual in the defined setting.” (Stolovitch & Keeps 1999) Justification for Choice of Problem Solution The problem solutions were carefully examined and proposed to the deputy superintendent. The first high cost solution, hiring of additional personnel was proposed but rejected as it is too costly and there are no additional funds at the State Department of Education level to provide for this option as the sate is suffering from budget cuts resulting in furlough days for existing staff. The high cost solution also still results in the provision of one staff member to cover the state district needs and ultimately would lead to this single staff person potentially receiving more than 8,000 email requests for service and another back log occurring. The state has more than 120 district level Testing Coordinators who, if trained, would provide more immediate services. . The second medium cost option of bringing district personnel into a central location for training is also too costly as there would be considerable travel involved and was also rejected. This second option would provide district level training and support from Testing Coordinators but all coordinators would have to be available during a specific time to attend FTF training. The likelihood that we could get all 120 testing coordinators free from their other district responsibilities would be slim as nearly all perform other job duties not related to their Testing Coordinator roles as in most of the medium and smaller districts the testing coordinators position is part time.. FTF training does not allow the participants to review or return to the training at will and is a one time event so was also deemed less convenient. The third low cost solution of providing district level training virtually, via the Idaho Training Clearinghouse via a webinar then archiving that webinar was deemed the most cost effective and convenient strategy as attendees can choose whether to attend the live training or to access the archived webinar at their convenience. The ITC already is the central repository for all Division of Assessment Alternate Assessment and I PASS training and materials for teachers so would provide a forum for this natural extension of training to IT and Testing Coordinators. This final option of virtual delivery of training and materials will actually be delivered at no extra cost to the State Department of Education as the Idaho Training Clearinghouse contract is covered by the Division of School Improvement and my salary is paid by the same division at no extra cost. Total cost to deliver and create training materials will be included in salaries already budgeted. Project Management Techniques Role as Project Manager In my role as project manager will be three fold; creator of training materials, trainer and ongoing support provider. As the author of all training materials I will produce a Quick Guide to I PASS for Testing Coordinators and IT staff. I will write and deliver a webinar which will be delivered online statewide. I will act as the central contact with district level IT and Testing Coordinators as well as teachers and administrators responding to questions regarding troubleshooting any questions or issues with all functions related to I PASS. My administrative assistant is also trained to take over the provision of a first tier of support prior to sending issues to me via email. Resource Management Techniques Including Resources and Tracking Tracking will be accomplished by the Idaho Training Clearinghouse website which has the capacity to record the attendees at webinars and these records will be matched with our SDE list of IT and Testing Coordinator Staff assuring all receive the opportunity for training. We have discussed making attendance mandatory. The other resources utilized will be existing personnel which is the Alternate Assessment Coordinator for Division of Assessment and the existing administrative assistant who has been trained to perform the following tasks; assigning students to classes, assessments to students and teachers to students. She is able to walk district personnel through these functions over the phone and via email in case the district level personnel still have difficulties performing the necessary functions within I PASS.
Delivery System Management Techniques and Medium for Intervention Delivery
Instruction requires the following components; information, knowledge, and skill acquisition are to improve knowledge and skills, training must be provided. Job aids can provide for informational needs. (Stolovitch and Keeps,1992) Job aids are those materials which support the user who has not memorized the training materials well enough to complete the required tasks. This project will also include the writing and publishing of an I PASS Quick Guide with screen shots which have been taken with Snag It software to create a step by step guide IT and Testing Coordinators can use as a resource to consult leaving the guide open as they work through all of the steps. Job aids which are designed appropriately with instructions and illustrations are a cost effective way to support trainees and include such materials as checklists, worksheets and well designed, job aids can augment FTF training by supporting trainees in what they learned but do not yet have memorized. (Mager, 1992) The delivery system for training will be webinar created via PowerPoint with embedded sound. The sound will be recorded via Audacity and very user friendly digital recording software. The webinar will be scheduled and then offered online to all district IT and Testing Coordinator staff. The training will facilitate the provision of support at the district rather than the state level help desk. The medium for delivery of assistance at the district level will be up to the district with the suggestion that some method for tracking of assistance be kept but no state level monitoring will be provided nor will the state provide the help desk tracking system or software. Email deliver of requests from teachers and administrators will be directed to the newly trained IT and or Testing Coordinators at the district level. The SDE will not be involved with monitoring the frequency or outcomes of these requests except that this first tier of support will filter all requests from the users forwarding only the more complex issues to the SDE, second tier which is the administrative assistant, third tier which is the Alternate Assessment Coordinator reserving the SDE IT department as the final fourth tier for only those purely technical issues which can not be resolved at the lower tiers.
Information Management Techniques Including; How Information is Stored, Accessed, and Processed, and Evaluation of the intervention The Idaho Training Clearinghouse (ITC) will be used for these informational needs including; storage, accessing training and materials as well as providing a continual forum for information exchange. The ITC has a page for the Alternate Assessment Learning Community maintained by the Alternate Assessment Coordinator. This learning community houses all training materials including archived webinars and podcasts, training materials such as Quick Guides and handbooks as well as the FAQ page where users can send in questions which are answered and posted as well as categorized for other users to view. The information will be stored here after live webinar attendance. The attendance list will be provided by them. At the end of the year IT and Testing Coordinators will be surveyed via Survey Monkey at the end of the school year to provide feedback regarding further training needs. The Division of Assessment already has a license to use Survey Monkey so there will be no additional training costs.
References
Mager, R. F. (1992). What every manager should know about training; or “I’ve got a training probem”…and other odd ideas. Belmont, CA Lake Stolovitch, H.D., & Keeps, E. J. (1999). What is human performance technology? In Handbook of human performance technology: Improving individual and organization performance worldwide (2nd ed.). San Francisco, CA: Jossey-Bass. Retrieved from http://web.utk.edu/~cis/hpt/hpt%20defined.pdf
Carol Scholz Organizational History and Background
1. Name of organization Idaho State Department of Education – Division of Assessment – Project - I-PASS (Individual Portfolio Artifact Submission System) 2. Summary Project - I-PASS (Individual Portfolio Artifact Submission System) is an online system which I have worked to develop with Caring Technologies of Boise and our own Idaho SDE dept. over the course of the last year to deliver the first in the nation, alternate assessment via a digital online portfolio. I-PASS allows teachers to upload evidence of student learning via video, digital photo or scanned or faxed student work into individual portfolios for each content area including Math, Reading, Language Usage and Science. This online assessment was developed to replace a former teacher rating scale which consisted of a single checklist for each content area. This digital portfolio was launched in May and we currently have more than 70,000 individual pieces of student work uploaded into our system including thousands of video. I have purchased 400 Flip cameras for teachers across the state and they have already received training via Webinar and Podcast on the use of the system as well as how to administer the assessment itself. All training and materials are available on the Idaho Training Clearing House at I also maintain a Learning Community there as well. Problem – Training District Information Technology Directors and Testing Coordinators Statewide. (Teacher Training Already Accomplished during 2009-2010) The problem that I have identified which I would like to address is the training of the district level Information Technology Directors as well as the district level Testing Coordinators statewide on the use of the I-PASS system and how they can best support these teachers. No training was provided because I was so focused on designing it, producing an assessment, publishing training materials and getting the system up and running by the deadline for teachers statewide that I did not have time to produce training materials for these key staff. I do not have a coworker who can assist with training or producing materials so this is the problem I would like to address prior to school opening in the Fall. To have a training component ready to go for both of these district level personnel. IT and Testing coordinators who both play separate roles in the delivery of this online system will necessitate the development of two separate training manuals as well as a means to deliver the training remotely via webinar or podcast etc. A support structure needs to be developed on a local level for teachers and as we will require testing coordinators to maintain the system at the local level. Training will need to be done so that they have the skills necessary to create online class lists updating them as needed with the correct content as well as setting up the hardware and networked software as well as computer access for teachers to use to conduct the assessment. 3. Background of Organization -
When was the organization founded? Where is it located?
The Idaho State Department of Education Division of Assessment was just organized this past summer. I came on board in July and a Deputy Superintendent was hired in august. It is a brand new division.
What is the mission or vision of the organization?
The Division of Assessment has not developed or published an official mission statement or goals. Here is what is on the web. The Division of Assessment will measure and improve student achievement, identify areas needing intervention, remediation or acceleration, inform parents and guardians about their child's progress, provide comparative local, state and national data regarding the achievement of students in essential skill areas, identify performance trends in student achievement across grade levels tested and student growth over time, and help to determine the technical assistance and consultation priorities for the State Department of Education.
What are the stated goals of the organization?
The Division of Assessment has not developed or published an official mission statement or goals. Unofficially the goal is to provide for the administration of all assessments including federally mandated as well as state mandated assessments to all students including ELL and special education students according to the NCLB and IDEA laws.
Who is the organization’s clientele?
All students in the state of Idaho.- The population of students with the most significant cognitive disabilities is 1% of the total population of special education students which is approximatly 1,500 students.
Who are the key personnel in the organization that can provide you with insights into the performance problem for which you will propose an intervention?
I will interview a testing coordinator as well as an information technology director at the district level to ascertain their views on how to best provide them with the training they need.
Does the organization have a Web site? If so, record the URL.
I-PASS (Individual Portfolio Academic Submission System) Coordinator Support
The identified problems are;
1. Testing Coordintors - District Testing coordinators do not know how to create class lists, assign tests and students so the SDE gets alot of help desk referrals.
2. Information Technology Leaders - District IT people do not know what district adjustments to the available technology are needed to help the teachers to be successful. There are firewall issues, speed issues, permission issues and software issues all of which arose after the online system was deployed and no IT training on I PASS specifically for them nor any way for these district IT to collaborate or share via a learning community.
I-PASS (Individual Portfolio Academic Submission System) Support Proposed Solutions
The Proposed Solutions are;
The low-cost strategies (at a cost of approximately 0 - $5,000)
Webinar - The lowest cost strategy is to produce a webinar and archive it on the Idaho Training Clearinghouse, which trains every testing coordinator in the state of Idaho (120) how to maintain the records for the ISAT-Alt (Idaho State Assessment Tests Alternate) using the I-PASS online digital portfolio system. The components of this training would be conducted via Webinar training then demo of the online system live followed by a Survey Monkey assessment. The cost would be included in the salary already paid for my position so in essence would be zero additional cost. \ NING - Another low cost solution would be to create a NING to deliver a webinar, training mateirals as well as video and a place for IT folks to meet in a learning community to share solutions and information amongst themselves.
Assigning Content Area Assessment Folders to Students Assigning Students to Teachers, Schools and Districts Assigning Teachers to Schools and Districts Monitoring these Assignments for Change Maintaining excel records of all students eligible to take the Alternate Assessment Working with teachers to upload student data. Troubleshooting simple technology issues to assist teachers. Monitoring the upload of data to assure completion of the assessment by every eligible student.
The medium-cost strategy would resolve the performance gap over several years’ time (at a cost of approximately $25,000)
Bring the testing coordinators to Boise for hands on training once per year on the same topics rather than have them view webinars or participate in a NING remotely. This would provide a hands-on experience where they could ask questions and receive assistance in addition to live demos. It would give them opportunities to network too.
The high-cost strategy “magic wand” to eliminate the performance gap cost of approximately $50,000).
Hire a state department dedicated IT-Testing Coordinator specifically assigned to maintaining the system and training at the district level. This would be the easiest for me as they would handle training and maintaining the system. Fat chance I don't think so! That's why its called a magic wand it is make believe. I am expected to do more with less not less with more.
I have rewritten the project based upon feedback from you and from Dr. Bauder on the module 3 assignments. I appreciate all of your suggestions and have incorporated them into this revision! So there are two verisons the revised one is first on this page and the module 3 version follows.
Thank you for your input I look forward to reading your responses!
With Warm Regards,
Carol
Modules One Through Four Summary
Organizational History and Background
Name of Organization and ProjectOrganization - Idaho State Department of Education – Division of Assessment
Project - Training Key Personnel for I-PASS (Individual Portfolio Artifact Submission System)
Project Summary
This project involves the training of key personnel for I-PASS (Individual Portfolio Artifact Submission System). I PASS is an online system which was created as a collaborative venture with Caring Technologies of Boise and the Idaho SDE department over the course of the last year to deliver the first in the nation, alternate assessment via a digital online portfolio. I-PASS allows teachers to upload evidence of student learning via video, digital photo or scanned or faxed student work into individual portfolios for each content area including Math, Reading, Language Usage and Science. This online assessment was developed to replace a former teacher rating scale which consisted of a single page checklist for each content area. This digital portfolio was launched in May and there are more than 70,000 individual pieces of student work uploaded into the I PASS system. Four hundred Flip cameras were purchased for teachers across the state to document student learning via video. Educators have already received training via Webinar and Podcast on the use of the system as well as how to administer the assessment itself. All training and materials are available on the Idaho Training Clearing House. An Alternate Assessment Learning Community is maintained there as well where FAQ, The scoring of these portfolios has just been completed online by teachers throughout the state, whom I have trained in a series of four regional workshops.
Problems
There are three identified problems.
The first is, that there will be no Idaho State Department of Education Information Technology support helpdesk staff to provide online or telephone support to districts this Fall on the I-PASS system. It is expected that responsibility for maintaining classes and providing support to teachers and staff will be transferred to the district level.
The second identified problem is related to the first. The district level IT and Testing Coordinators have not been trained to perform the support tasks necessary to assist district teachers statewide on the use of the I-PASS system.
The third problem is that the training and delivery has not be created for this transfer of responsibility, as this is a newly developed system we created ourselves so does not come with documentation or support materials for training.
It has become evident that a support structure needs to be developed on a state-wide and local level for teachers through these key personnel at the district level. District IT staff and Testing Coordinators will be required to maintain the system at the local level. Training will need to be done so that they have the skills necessary to maintain online class lists, updating them as needed with the correct content, setting up the hardware, networked software and computer access for teachers to use to conduct the assessment.
3. Background of Organization
When was the organization founded? Where is it located?
The Idaho State Department of Education Division of Assessment was just organized this past summer as a new division within the Idaho State Department of Education. It was formerly housed within the Idaho State Board of Education. It is housed at the SDE in Boise, Idaho. Two new coordinators and a new Deputy Superintendant were hired coming on board in July and August. It is a brand new division working together for one year.
What is the mission or vision of the organization?
The Division of Assessment has not developed or published an official mission statement. Here is what is on the web. The Division of Assessment will measure and improve student achievement, identify areas needing intervention, remediation or acceleration, inform parents and guardians about their child's progress, provide comparative local, state and national data regarding the achievement of students in essential skill areas, identify performance trends in student achievement across grade levels tested and student growth over time, and help to determine the technical assistance and consultation priorities for the State Department of Education.
What are the stated goals of the organization?
The Division of Assessment has not developed or published goals. Unofficially the goal is to provide for the administration of all assessments including federally mandated as well as state mandated assessments to all students including ELL and special education students according to the NCLB and IDEA federal guidelines and laws.
Who is the organization’s clientele?
The identified clientele of the Department of Education includes; students, educators and administrators throughout the state of Idaho. The specific identified clientele for the I PASS project are those students with the most significant cognitive disabilities and their teachers. This student population is 1% of the total population of special education students at approximately 1,500 students who have been identified on their IEP as eligible to take the Idaho Standards Achievement Tests Alternate (ISAT – Alt). This is the federally mandated NCLB and IDEA state assessment corresponding to the ISAT itself designed for the general education population.
Who are the key personnel in the organization that can provide you with insights into the performance problem for which you will propose an intervention?
The key personnel with insight into the performance issue, is the Deputy Superintendent of the Assessment Division. Final decisions regarding state wide training and service are made in collaboration with her, After discussion with the Deputy Superintendent, it was reported that IT would no longer be able to provide support directly to district teachers. The SDE would provide the initial data entry of district provided demographics such as student, teacher, assessment specifics at the beginning of the school year but that districts would have to maintain class lists including adding, removing students and teachers, associating specific assessment folders with students and providing technical assistance for uploading data.
The Testing Coordinators and IT staff, at the district level, also provided the identification of the problem as well as for the justification for the choice of solution. Rather than poll these staff regarding their individual needs, email records were reviewed indicating that over 8,000 emails were received from educators requesting help with I PASS by state IT department personnel. In addition the district level IT and Testing Coordinators have expressed a need for further training and support directly to the Deputy Superintendent.
The results of the interview also revealed that there is no funding to hire another administrative level person at the SDE assessment or IT division levels..
Does the organization have a Web site? If so, record the URL.
http://www.sde.idaho.gov/
Do you have access to any public relations materials or annual reports for the organization? Where can you obtain these?
http://www.sde.idaho.gov/ General website for State Department of Education
http://www.sde.idaho.gov/site/assessment/ISAT/results.htm ISAT test results for Idaho
http://itcnew.idahotc.com/dnn/alternate-assessment.aspx Idaho Training Clearinghouse where I maintain a Learning Community and publish all WebEx, podcast and publications
I-PASS (Individual Portfolio Academic Submission System) Support Project
Human Performance Technology (HPT) attempts to initiate changes to a system to improve it in terms of the achievements that have been identified as critical to success.
In utilizing the HPT model on the implementation of I PASS and in the review of the success of the previous year, the focus will be on these outcomes and accomplishments which have been identified as critical to performance. The procedures identified to ameliorate the problems will be applied in a methodical manner in identifying the performance gap, indentifying appropriate interventions and in their application to improve performance. (Stolovitch & Keeps 1999).
The identified problems are;
There are three identified problems.
The first is, that there will be no Idaho State Department of Education Information Technology support helpdesk staff to provide online or telephone support to districts this Fall on the I-PASS system. It is expected that responsibility for maintaining classes and providing support to teachers and staff will be transferred to the district level.
The second identified problem is related to the first. The district level IT and Testing Coordinators have not been trained to perform the support tasks necessary to assist district teachers statewide on the use of the I-PASS system.
The third problem is that the training and delivery has not be created for this transfer of responsibility, as this is a newly developed system we created ourselves so does not come with documentation or support materials for training.
District Testing coordinators do not know how to create class lists, assign tests and students so the SDE gets a lot of help desk referrals to correct errors or make changes as students move and change schools, teachers and districts. A statewide information system is in place to establish the initial class, school and district list but is riddled with data errors which must be corrected manually as they arise. The data is only as good as the accuracy of the person who enters it and there are bound to be errors. The volume of these data errors and constant changes in assignment of individual students is so extensive despite expert assistance, the capacity of the SDE helpdesk hat been stretched to its limit and will not be able to continue to handle this increased demand. There will be a push out to districts for IT and Testing Coordinators to maintain these needed adjustments to the initial data uploaded in October.
I-PASS (Individual Portfolio Academic Submission System) Support Proposed Solutions
The low-cost strategy (at a cost of approximately 0 - $5,000) is to produce a webinar and archive it on the Idaho Training Clearinghouse, which trains every testing coordinator in the state of Idaho (120) how to maintain the records for the ISAT-Alt (Idaho State Assessment Tests Alternate) using the I-PASS online digital portfolio system. The components of this training would be conducted via Webinar training then demo of the online system live followed by a Survey Monkey assessment. The cost would be included in the salary already paid for my position so in essence would be zero additional cost.
Another low cost solution would be to create a NING to deliver a webinar, training materials as well as video. The advantage to developing a NING is it would provide for a place for IT folks to meet in a learning community to share solutions and information amongst themselves.
Tech Talk – Another free format for communicating with IT district personell is through the portal Tech Talk. It provides for a platform for blogging about various subjects and is a secure site providing for a free forum for concerns and information without providing public access.
The most practical cost effective solution for these problems may be to use a combination of the two free services; Idaho Training Clearinghouse and Tech Talk. Neither provides for all of the necessary components for great training but together may be adequate to get the job done in tough economic times. NING just announced that it will charge beginning this month so this option is probably not going to work out but probably would provide the best platform.
Necessary Training for Testing Coordinators Includes;
Assigning Content Area Assessment Folders to Students
Assigning Students to Teachers, Schools and Districts
Assigning Teachers to Schools and Districts
Monitoring these Assignments for Change
Maintaining excel records of all students eligible to take the Alternate Assessment
Working with teachers to upload student data.
Maintaining and providing the necessary hardware and software for I PASS
Troubleshooting simple technology issues to assist teachers.
Monitoring the upload of data to assure assessment completion of eligible students.
The Identified Possible Solutions
In the application of the HPT model “procedures derived from scientific research and
professional experience to the solution of practical problems”. (Stolovitch & Keeps 1999). HPT is systematic in that it is an organized approach to apply scientific research methodology and results to identify, define and analyze performance gaps then apply the interventions in a methodical manner to achieve the desired improvement in human performance of both the Assessment Coordinator and the Testing Coordinators as well as the IT personnel at the district level. apply them in a controlled manner. Documented evidence will be created via Survey Monkey both prior to implementation and subsequent to it to identify areas of need, knowledge base and potential solutions then to analyze results of the applied t raining solutions.
The medium-cost strategy would resolve the performance gap over several years’ time (at a cost of approximately $25,000). This medium cost strategy would be a combination of online delivery of training and support and live face to face. This would be accomplished by bringng the testing coordinators and IT professionals to Boise for hands on training once per year on the same topics rather than have them view webinars on the Idaho Training Clearinghouse, or participate in a NING and Tech Talk remotely . This would provide a hands-on experience where they could ask questions and receive assistance in addition to live demos. It would give them opportunities to network too as they are often isolated in remote districts and do not have access to a live learning community.
The high-cost strategy “magic wand” to eliminate the performance gap cost of approximately $50,000). This high cost strategy could be to hire a state department staff person dedicated to IT and Testing Coordinator Support specifically assigned to maintaining the I PASS system, Another high cost strategy would be to provide funds for the SDE to implement a state-wide help desk tracking system accessible to teachers, testing coordinators and It personnel across the state. This would provide support and training at the state and district level. They would handle training and maintaining the system and their position would separate out the duties of from training on assessment and supporting teachers in the classroom and the maintenance of the actual system IT and Testing Coordinator support itself. I don't think it is feasible. That's why it is called a magic wand solution! It is make believe. It is the expectation in tough economic times to do more with less not less with more.
Throughout this process of designing training to ameliorate the indentified problems the Human Performance and Technology model will be utilized to deal with the performance of the Alternate Assessment Coordinator, district information technology personnel and the district level testing coordinators. The effort will be to search for the most effective and efficient way to improve performance but accomplish this in the most cost-efficient way
The focus will be to”enhance individual human performance by looking at the organization, the process and the individual in the defined setting.” (Stolovitch & Keeps 1999)
Justification for Choice of Problem Solution
The problem solutions were carefully examined and proposed to the deputy superintendent.
The first high cost solution, hiring of additional personnel was proposed but rejected as it is too costly and there are no additional funds at the State Department of Education level to provide for this option as the sate is suffering from budget cuts resulting in furlough days for existing staff. The high cost solution also still results in the provision of one staff member to cover the state district needs and ultimately would lead to this single staff person potentially receiving more than 8,000 email requests for service and another back log occurring. The state has more than 120 district level Testing Coordinators who, if trained, would provide more immediate services. .
The second medium cost option of bringing district personnel into a central location for training is also too costly as there would be considerable travel involved and was also rejected. This second option would provide district level training and support from Testing Coordinators but all coordinators would have to be available during a specific time to attend FTF training. The likelihood that we could get all 120 testing coordinators free from their other district responsibilities would be slim as nearly all perform other job duties not related to their Testing Coordinator roles as in most of the medium and smaller districts the testing coordinators position is part time.. FTF training does not allow the participants to review or return to the training at will and is a one time event so was also deemed less convenient.
The third low cost solution of providing district level training virtually, via the Idaho Training Clearinghouse via a webinar then archiving that webinar was deemed the most cost effective and convenient strategy as attendees can choose whether to attend the live training or to access the archived webinar at their convenience. The ITC already is the central repository for all Division of Assessment Alternate Assessment and I PASS training and materials for teachers so would provide a forum for this natural extension of training to IT and Testing Coordinators. This final option of virtual delivery of training and materials will actually be delivered at no extra cost to the State Department of Education as the Idaho Training Clearinghouse contract is covered by the Division of School Improvement and my salary is paid by the same division at no extra cost. Total cost to deliver and create training materials will be included in salaries already budgeted.
Project Management Techniques Role as Project Manager
In my role as project manager will be three fold; creator of training materials, trainer and ongoing support provider. As the author of all training materials I will produce a Quick Guide to I PASS for Testing Coordinators and IT staff. I will write and deliver a webinar which will be delivered online statewide. I will act as the central contact with district level IT and Testing Coordinators as well as teachers and administrators responding to questions regarding troubleshooting any questions or issues with all functions related to I PASS. My administrative assistant is also trained to take over the provision of a first tier of support prior to sending issues to me via email.
Resource Management Techniques Including Resources and Tracking
Tracking will be accomplished by the Idaho Training Clearinghouse website which has the capacity to record the attendees at webinars and these records will be matched with our SDE list of IT and Testing Coordinator Staff assuring all receive the opportunity for training. We have discussed making attendance mandatory. The other resources utilized will be existing personnel which is the Alternate Assessment Coordinator for Division of Assessment and the existing administrative assistant who has been trained to perform the following tasks; assigning students to classes, assessments to students and teachers to students. She is able to walk district personnel through these functions over the phone and via email in case the district level personnel still have difficulties performing the necessary functions within I PASS.
Delivery System Management Techniques and Medium for Intervention Delivery
Instruction requires the following components; information, knowledge, and skill acquisition are to improve knowledge and skills, training must be provided. Job aids can provide for informational needs. (Stolovitch and Keeps,1992) Job aids are those materials which support the user who has not memorized the training materials well enough to complete the required tasks. This project will also include the writing and publishing of an I PASS Quick Guide with screen shots which have been taken with Snag It software to create a step by step guide IT and Testing Coordinators can use as a resource to consult leaving the guide open as they work through all of the steps. Job aids which are designed appropriately with instructions and illustrations are a cost effective way to support trainees and include such materials as checklists, worksheets and well designed, job aids can augment FTF training by supporting trainees in what they learned but do not yet have memorized. (Mager, 1992) The delivery system for training will be webinar created via PowerPoint with embedded sound. The sound will be recorded via Audacity and very user friendly digital recording software. The webinar will be scheduled and then offered online to all district IT and Testing Coordinator staff. The training will facilitate the provision of support at the district rather than the state level help desk. The medium for delivery of assistance at the district level will be up to the district with the suggestion that some method for tracking of assistance be kept but no state level monitoring will be provided nor will the state provide the help desk tracking system or software. Email deliver of requests from teachers and administrators will be directed to the newly trained IT and or Testing Coordinators at the district level. The SDE will not be involved with monitoring the frequency or outcomes of these requests except that this first tier of support will filter all requests from the users forwarding only the more complex issues to the SDE, second tier which is the administrative assistant, third tier which is the Alternate Assessment Coordinator reserving the SDE IT department as the final fourth tier for only those purely technical issues which can not be resolved at the lower tiers.
Information Management Techniques Including; How Information is Stored, Accessed, and Processed, and Evaluation of the intervention
The Idaho Training Clearinghouse (ITC) will be used for these informational needs including; storage, accessing training and materials as well as providing a continual forum for information exchange. The ITC has a page for the Alternate Assessment Learning Community maintained by the Alternate Assessment Coordinator. This learning community houses all training materials including archived webinars and podcasts, training materials such as Quick Guides and handbooks as well as the FAQ page where users can send in questions which are answered and posted as well as categorized for other users to view. The information will be stored here after live webinar attendance. The attendance list will be provided by them. At the end of the year IT and Testing Coordinators will be surveyed via Survey Monkey at the end of the school year to provide feedback regarding further training needs. The Division of Assessment already has a license to use Survey Monkey so there will be no additional training costs.
References
Mager, R. F. (1992). What every manager should know about training; or “I’ve got a training probem”…and other odd ideas. Belmont, CA Lake
Stolovitch, H.D., & Keeps, E. J. (1999). What is human performance technology? In Handbook of human performance technology: Improving individual and organization performance worldwide (2nd ed.). San Francisco, CA: Jossey-Bass. Retrieved from http://web.utk.edu/~cis/hpt/hpt%20defined.pdf
Carol Scholz Organizational History and Background
1. Name of organizationIdaho State Department of Education – Division of Assessment – Project - I-PASS (Individual Portfolio Artifact Submission System)
2. Summary
Project - I-PASS (Individual Portfolio Artifact Submission System) is an online system which I have worked to develop with Caring Technologies of Boise and our own Idaho SDE dept. over the course of the last year to deliver the first in the nation, alternate assessment via a digital online portfolio. I-PASS allows teachers to upload evidence of student learning via video, digital photo or scanned or faxed student work into individual portfolios for each content area including Math, Reading, Language Usage and Science. This online assessment was developed to replace a former teacher rating scale which consisted of a single checklist for each content area. This digital portfolio was launched in May and we currently have more than 70,000 individual pieces of student work uploaded into our system including thousands of video. I have purchased 400 Flip cameras for teachers across the state and they have already received training via Webinar and Podcast on the use of the system as well as how to administer the assessment itself. All training and materials are available on the Idaho Training Clearing House at I also maintain a Learning Community there as well.
Problem – Training District Information Technology Directors and Testing Coordinators Statewide. (Teacher Training Already Accomplished during 2009-2010)
The problem that I have identified which I would like to address is the training of the district level Information Technology Directors as well as the district level Testing Coordinators statewide on the use of the I-PASS system and how they can best support these teachers. No training was provided because I was so focused on designing it, producing an assessment, publishing training materials and getting the system up and running by the deadline for teachers statewide that I did not have time to produce training materials for these key staff. I do not have a coworker who can assist with training or producing materials so this is the problem I would like to address prior to school opening in the Fall. To have a training component ready to go for both of these district level personnel. IT and Testing coordinators who both play separate roles in the delivery of this online system will necessitate the development of two separate training manuals as well as a means to deliver the training remotely via webinar or podcast etc.
A support structure needs to be developed on a local level for teachers and as we will require testing coordinators to maintain the system at the local level. Training will need to be done so that they have the skills necessary to create online class lists updating them as needed with the correct content as well as setting up the hardware and networked software as well as computer access for teachers to use to conduct the assessment.
3. Background of Organization -
- When was the organization founded? Where is it located?
The Idaho State Department of Education Division of Assessment was just organized this past summer. I came on board in July and a Deputy Superintendent was hired in august. It is a brand new division.- What is the mission or vision of the organization?
The Division of Assessment has not developed or published an official mission statement or goals. Here is what is on the web.The Division of Assessment will measure and improve student achievement, identify areas needing intervention, remediation or acceleration, inform parents and guardians about their child's progress, provide comparative local, state and national data regarding the achievement of students in essential skill areas, identify performance trends in student achievement across grade levels tested and student growth over time, and help to determine the technical assistance and consultation priorities for the State Department of Education.
- What are the stated goals of the organization?
The Division of Assessment has not developed or published an official mission statement or goals. Unofficially the goal is to provide for the administration of all assessments including federally mandated as well as state mandated assessments to all students including ELL and special education students according to the NCLB and IDEA laws.- Who is the organization’s clientele?
All students in the state of Idaho.- The population of students with the most significant cognitive disabilities is 1% of the total population of special education students which is approximatly 1,500 students.- Who are the key personnel in the organization that can provide you with insights into the performance problem for which you will propose an intervention?
I will interview a testing coordinator as well as an information technology director at the district level to ascertain their views on how to best provide them with the training they need.- Does the organization have a Web site? If so, record the URL.
http://www.sde.idaho.gov/- Do you have access to any public relations materials or annual reports for the organization? Where can you obtain these?
http://www.sde.idaho.gov/General website for State Department of Education
http://www.sde.idaho.gov/site/assessment/ISAT/results.htm
ISAT test results for Idaho
http://itcnew.idahotc.com/dnn/alternate-assessment.aspx
Idaho Training Clearinghouse where I maintain a Learning Community and publish all webex, podcast and publications
I-PASS (Individual Portfolio Academic Submission System) Coordinator Support
The identified problems are;
1. Testing Coordintors - District Testing coordinators do not know how to create class lists, assign tests and students so the SDE gets alot of help desk referrals.
2. Information Technology Leaders - District IT people do not know what district adjustments to the available technology are needed to help the teachers to be successful. There are firewall issues, speed issues, permission issues and software issues all of which arose after the online system was deployed and no IT training on I PASS specifically for them nor any way for these district IT to collaborate or share via a learning community.
I-PASS (Individual Portfolio Academic Submission System) Support Proposed Solutions
The Proposed Solutions are;
- The low-cost strategies (at a cost of approximately 0 - $5,000)
Webinar - The lowest cost strategy is to produce a webinar and archive it on the Idaho Training Clearinghouse, which trains every testing coordinator in the state of Idaho (120) how to maintain the records for the ISAT-Alt (Idaho State Assessment Tests Alternate) using the I-PASS online digital portfolio system. The components of this training would be conducted via Webinar training then demo of the online system live followed by a Survey Monkey assessment. The cost would be included in the salary already paid for my position so in essence would be zero additional cost. \NING - Another low cost solution would be to create a NING to deliver a webinar, training mateirals as well as video and a place for IT folks to meet in a learning community to share solutions and information amongst themselves.
Assigning Content Area Assessment Folders to Students
Assigning Students to Teachers, Schools and Districts
Assigning Teachers to Schools and Districts
Monitoring these Assignments for Change
Maintaining excel records of all students eligible to take the Alternate Assessment
Working with teachers to upload student data.
Troubleshooting simple technology issues to assist teachers.
Monitoring the upload of data to assure completion of the assessment by every eligible student.
- The medium-cost strategy would resolve the performance gap over several years’ time (at a cost of approximately $25,000)
Bring the testing coordinators to Boise for hands on training once per year on the same topics rather than have them view webinars or participate in a NING remotely. This would provide a hands-on experience where they could ask questions and receive assistance in addition to live demos. It would give them opportunities to network too.- The high-cost strategy “magic wand” to eliminate the performance gap cost of approximately $50,000).
Hire a state department dedicated IT-Testing Coordinator specifically assigned to maintaining the system and training at the district level. This would be the easiest for me as they would handle training and maintaining the system. Fat chance I don't think so! That's why its called a magic wand it is make believe. I am expected to do more with less not less with more.e content of your page here.